iGesture: A Platform for Investigating Multimodal, Multimedia Gesture-based Interactions
نویسندگان
چکیده
This paper introduces the iGesture platform for investigating multimodal gesture based interactions in multimedia contexts. iGesture is a low-cost, extensible system that uses visual recognition of hand movements to support gesturebased input. Computer vision techniques support gesture based interactions that are lightweight, with minimal interaction constraints. The system enables gestures to be carried out ‘in the environment’ at a distance from the camera, enabling multimodal interaction in a naturalistic, transparent manner in a ubiquitous computing environment. The iGesture system can also be rapidly scripted to enable gesturebased input with a wide variety of applications. In this paper we present the technology behind the iGesture software, and a performance evaluation of the gesture recognition subsystem. We also present two exemplar multimedia application contexts which we are using to explore ambient gesture-based interactions.
منابع مشابه
iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms
Existing gesture recognition tools and frameworks tend to focus on specific settings, gesture sets or algorithms. Further, they are often designed to support the developers of either applications or algorithms, but not both. Our goal was to develop a general and extensible framework that provides an integrated platform for the design and evaluation of algorithms, as well as for their deployment...
متن کاملMultimodal and Cross-modal Processing in Interactive Systems Based on Tangible Acoustic Interfaces
This paper presents some recent developments at DISTInfoMus Lab on multimodal and cross-modal processing of multimedia data streams with a particular focus on interactive systems exploiting Tangible Acoustic Interfaces (TAIs). In our research multimodal and cross-modal algorithms are employed for enhancing the extraction and analysis of the expressive information conveyed by gesture in non-verb...
متن کاملO’Halloran, K
The following paper discusses the challenges faced in designing a flexible, easy-to-use, cross-platform software tool for modeling, analyzing and retrieving meaning from multimedia data. In a project currently being undertaken at the Multimodal Analysis Lab, Interactive and Digital Media Institute (IDMI), at the National University of Singapore (see http://multimodal-analysis-lab.org/), social ...
متن کاملEvaluating the Social Acceptability of Multimodal Mobile Interactions
Multimodal mobile interfaces require users to adopt new and possibly strange behaviors in public places. It is important to design these interfaces to account for the social restrictions of public settings. However, past research in multimodal interaction has primarily focused on issues of sensing and recognition rather than the investigation of user opinions and social factors that influence t...
متن کاملMultimodal Analysis of Expressive Gesture in Music and Dance Performances
This paper presents ongoing research on the modelling of expressive gesture in multimodal interaction and on the development of multimodal interactive systems explicitly taking into account the role of non-verbal expressive gesture in the communication process. In this perspective, a particular focus is on dance and music as first-class conveyors of expressive and emotional content. Research ou...
متن کامل